Learned Models of Performance for Many Planners

We describe a large scale study of planners and their performance: 28 planners on 4726 benchmark problems. In the first part of the paper, we apply off-the-shelf machine learning techniques to learn models of the planners’ performance from the data. In the evaluation of these models, we address the critical question of whether accurate models can be learned from easily extractable problem features. In the second part, we show how the models can be useful to furthering planner performance and understanding. We offer two contributions: 1) We demonstrate that accurate models of runtime and probability of success can be learned using off-the-shelf machine learning techniques, and 2) We show that the learned models can be leveraged to support a planner portfolio which improves over individual planners. We also discuss how the models can be analyzed to better understand how planner design decisions contribute to their performance.